Software for assessment and monitoring the decoding skills development of children from the elementary school: validity based on response process

ABSTRACT Purpose To continue the validation process of the Decoding Development Monitoring Protocol (PRADE) in software format in the validity evidence stage based on response processes. Methods 250 individuals participated in this study, 125 individuals from private schools and 125 individuals from public schools. The assessment was carried out in person using the software that hosts the instrument's tasks, which are organized into decoding linguistically balanced words and non-words, respecting the decoding rules of Brazilian Portuguese. The software prepares an individual performance report for each participant, counting the decoding time for each stimulus, as well as the number of words decoded correctly. The data is organized considering the correct decoding time of the stimuli, decoding accuracy and percentage of correct answers. All data underwent statistical analysis using SPSS software. Results The data indicated an important effect of the length of words and non-words on public and private school students. Furthermore, it was possible to observe the evolution of decoding, depending on the school year, in all the variables studied. In both groups, a strong influence of non-words on student performance throughout Elementary School I was observed. Conclusion The data indicate validity in the analysis of response processes, since it was possible to adequately characterize the performance of school children public and private throughout Elementary School I, characterizing each group, as well as their differences according to the advancement of schooling.


INTRODUCTION
The assessment of decoding fluency through read-aloud practice is the most common method to monitor children's acquisition and progress, both regarding school evaluation reports as well as to verify the effectiveness of the stimulation and/or intervention programs (1,2) .In fact, oral decoding assessment results are so important that they are considered to be fundamental predictors of the individual's reading performance.In the United States, for instance, oral reading fluency assessment scores are analyzed by the Federal Education Department to monitor the children's academic development and for the improvement of early stimulation programs (1,2) .
Oral reading of isolated words is undoubtedly the most widely used task to assess an individual's decoding proficiency.This is partially due to the fact that this specific task isolates the contextual or visual (pictorial) cues, therefore, the evaluation is strictly restrained to the decoding skill (2) .Reading assessment instruments are strongly based on the dual-route model, since they use words and non-words (2) .It is important to emphasize that efforts have been made to create and validate reading tests for Brazilian Portuguese (BP) speakers (3)(4)(5)(6) , however, most of them are more focused on reading and its processes instead of isolated decoding, not to mention the insufficient investigation on the tests' psychometric properties (7)(8)(9) .
According to the literature (10) , children who present any delay or fail to acquire the automatic word recognition skill may be at risk for several alterations in written language learning.Nevertheless, despite the advances in relation to the study of reading regarding its parameters and underlying skills, the number of validated instruments for an effective evaluation and intervention of potential reading and learning disorders is still reduced in Brazil (3)(4)(5)(6)(7)(8)(9) .That being said, most of these instruments are not in a software or application format, which not only brings the decoding assessment process into the twenty-first century, but can also provide a more reliable data concerning the decoding time, the number/percentage of correct answers or accuracy, and the number of words read correctly per minute or in one minute, which are all indicators reckoned as the gold standard in international decoding tests.The EduEdu® application is an exception in this case, providing activities that allow to evaluate and monitor the learning of the written language in a holistic approach, despite not having the appropriate scientific validation (11) .
In a country with an educational reality such as Brazil's, in which it is recurrent for students to experience some degree of difficulty reading or writing, it is essential to characterize the children's reading skill earlier on, especially in the learning phase of decoding.This takes into consideration the latest report of the Programme for International Student Assessment -PISA, in which Brazil continues to exhibit high rates of school failure (12) .According to published data, the country has remained among the worst-performing countries for more than a decade.Furthermore, the abovementioned report indicates that Brazilian schoolchildren's reading difficulties usually onset in basic education, that is, related to the decoding competence, interfering with the consolidation of literacy.When misidentified, these difficulties can become chronic, triggering a low performance throughout the student's school career (12) .
As previously stated, the lack of properly validated instruments prevents professionals that are directly involved in working with schoolchildren (speech language therapists, teachers and psychologists) from establishing adequate parameters to identify the decoding difficulties and alterations, consequently hindering the development of more effective stimulation strategies, including in the classroom (10) .This reality is quite different in developed countries, where there is a standard use of validated tests to assess decoding, fluency and reading comprehension.Conventionally, these instruments are composed of a list of words and texts that must be read by the children at a predetermined interval of time (10,13,14) .
Hence, the elaboration of a Brazilian decoding assessment instrument that could be used as a tool to monitor this skill's development is fundamental for both the clinical and the educational fields.Moreover, it is extremely necessary to have such an instrument in a software format, in which performance can be measured more reliably, being suitably validated with the psychometric characteristics analyzed and with free access for speech language therapists, educational psychologists, parents, teachers or further professionals who work in learning.This way is possible to build a broad support network and supervise the decoding development, which is characterized by a multidisciplinary approach along with the construction of stronger relationships between professionals and the child's family (15) .It is crucial to highlight that monitoring instruments should be applied to healthy as well as to unhealthy individuals, targeting the identification of those presenting risk factors for certain conditions or disabilities, so that the early identification can produce better results (16) .To be effective, the instrument must be validated, be straightforward and have a simple application, reproducibility and accuracy (16) .
Consequently, the objective of the present study is to proceed with the validation process of the Decoding Development Monitoring Protocol (Protocolo de Acompanhamento do Desenvolvimento da Decodificação -PRADE), in a software format, regarding the validity evidence based on the response processes, since all previous steps have already been published and referenced (17)(18)(19) .

METHOD
This research was approved by the Research Ethics Committee of the University of São Paulo Medical School (Faculdade de Medicina da Universidade de São Paulo -FMUSP) (REC No. 2,262,300).This is a prospective study conducted in accordance with the principles of the Standards for Educational and Psychological Testing (SEPT) (20) , a guideline proposed by American organizations that compiles recommendations and fundamental definitions regarding the psychometric aspects involved from the preparation to the interpretation of the tests, accompanied by the different steps necessary to validate an instrument.The data collection procedures herein only began after the signing of the Informed Consent Form both by the school involved in the study and by the parents/guardians, in addition to the signing of the Consent Form by the children.

Case study
250 individuals participated in this study, 125 individuals from private schools and 125 individuals from public schools.Both groups were subdivided into five equal sized (No. = 25) groups according to the school year, that is, from the first to the fifth school year.To be included in the present study, children were required to meet the following criteria: be regularly enrolled in Elementary School; absence of complaints/ indicators of hearing or visual alterations; absence of indications of neurological or cognitive disorders; absence of retention in the school record; absence of phonological and oral language alterations, as verified through speech-language screening.
In order to ensure that the study sample comprised children with different academic profiles and to avoid singling out only one profile of children screened for participation, whether those with better academic performance or those with a greater difficulty to learn decoding, it was decided to use a stratified random sampling for the participant selection.Therefore, the children were initially numbered from 1 to 150, in ascending order according to the school year, in both schools.Then, these numbers were used to randomly select the final study sample, as characterized below.
As a result, the public school group was composed as follows: 25 1st grade children (12

Process
Considering that the present study is part of the PRADE validation process, it is important to emphasize that the following steps of validity evidence based on the test contents; delimitation of the target population; elaboration of the items; analysis of judges with expertise in the area; determination of the sample size; protocols to verify that the population understands the test items; application of the test in a sample of the target population; data analysis and correlations were carried out in a previous study and published in an international journal (17) confirming the validity of these procedures.
Under these circumstances, it was possible to proceed to the stage of validity evidence based on the response processes, which is characterized by the assessment of different strata of the target population's performance in the task application, seeking to understand the processes involved in the response patterns (21) .It is worth noting that phases of this stage have already been performed in previous studies with children presenting Developmental Language Disorder (DLD), which analyzed this population's performance in the test presented herein, as reported in two studies published in journals indexed within the Web of Science with a relevant impact factor (18,19) .The data suggested a difference in the performance of subjects with DLD when compared to their neurotypical peers, indicating an important discriminant validity of this instrument regarding neurotypical children and those with DLD.However, it is necessary to confirm this last aspect in future studies with different populations.
It should also be pointed out that, when it comes to preparing the materials to verify aspects of written language in Brazil, there is a history of poor performance in both national and international assessments, along with the literacy rates and alarming functional illiteracy that should also be taken into account, especially pertaining students from public schools and those in the intersectionality of public school added to social vulnerability.In this sense, it is valid to further verify the effectiveness of this instrument to adequately characterize the performance of public and private school students.
PRADE consists of linguistically balanced words elaborated according to the decoding rules of BP (22) , respecting the word length variation from mono to polysyllables appropriate to children in this school level (23) .Furthermore, the test has non-words that were derived from real words, likewise following the BP decoding rules and the variation from mono to polysyllables (23) .
The assessment was performed in person using the software that hosts PRADE's tasks, Psychopy®, on the participants' extracurricular time.It is noteworthy that the data collection occurred in the second semester of the school year, as this ensures that the academic skill profile is consistent with the individuals' school year, especially with regard to students in the first school year.When starting the software, a home screen appears welcoming the participant and explaining how the test works (Figure 1).
The task begins with real words that are randomly presented in arial font, uppercase, No. 20 written in white on a screen with a gray background (Figure 2).
During the assessment, the evaluator presses the number 0 for each wrong decoding response and the number 1 for each correct decoding.It should be made clear that "correct decoding" was considered to be the response consistent with the graphemephoneme correspondences, spelling rules and adequate tonicity in stimuli with diacritical marks (22) .Subsequently, the decoding of the non-words starts by showing the participant a new screen presenting the new instructions (Figure 3).
The procedure for evaluating the decoding of non-words is analogous to the one described for word decoding.It must be stressed that, for the characterization of the non-words' correct or incorrect decoding, it was strictly considered the participant's ability to use BP decoding rules, including those that contained some diacritical mark (22) .Specifically, the evaluator read the on-screen instructions to the participants to ensure that the task was fully understood.Ultimately, each child took an average of 2 minutes and 30 seconds to complete the entire task.
The software used herein produces an individual performance report for each participant, accounting for the decoding time of each stimulus, from monosyllables to polysyllables, along with the number correctly decoded words.Next, the data is tabulated considering the correct word decoding time, the percentage of correct answers per stimulus length (from mono to polysyllable) and their total scores, as well as the decoding accuracy, namely, the number of words/non-words correctly decoded per minute.The stimulus length and its total scores were also considered, for both words and non-words.It is essential to highlight that, in the analysis of the correct words and non-words decoding time, participants who did not correctly decode any stimulus were excluded.All data were statistically analyzed using the SPSS Statistics software, version 28.0 (IBM Corp., Armonk, NY, USA).

RESULTS
Table 1 depicts the central tendency and dispersion measures of the decoding time for each type of school according to the school year, word length and stimulus type; word/non-word.The data    indicate a greater number of children in the first year of private schools who were able to effectively decode words and non-words, the number decreases as for public school participants depending on the stimulus size.The influence of the stimulus length is evident in further school years, showing a closer approximation of public and private school participants' decoding time, as well as an increase in the number of children who were able to perform the task from the second year onwards.It was demonstrated that, following the course of the ensuing schooling process, both the number of participants and the proximity of the correct decoding times reaches an equivalence point, stabilizing in all participants from the third school year onwards.
Table 2 presents the central tendency and dispersion measures related to the correct answer percentage for each school type according to the school year, word length and of stimulus type; word/non-word.Once again, the data corroborates the influence of the stimulus length on the correct answer percentage for children from both schools, through all school years, especially concerning non-words.It is notable that private school students exhibit a higher correct answer percentage through all school years and for all stimuli lengths.The data also indicate the similarity of the students' performance from the third school year onwards, indicating a performance stabilization in later school years.
Table 3 shows the central tendency and dispersion measures of the decoding accuracy for each school type corresponding to the school year, stimulus length and type; word/non-word.The data suggests a better decoding accuracy of the private school students in both stimuli and in all variables as well as school year.Moreover, it is fundamental to emphasize that the results show a strong effect of word length for all school years, with a more pronounced influence of such effect on non-words.

DISCUSSION
The objective of the present study was to continue the validation process of the Decoding Development Monitoring Protocol (Protocolo de Acompanhamento do Desenvolvimento da Decodificação -PRADE) in a software format for the validity evidence stage based on the response processes, with children from public and private schools as participants.
The data evidences that the software used herein confirms the response processes' analysis validity, since it was possible to adequately characterize the performance of children from public and private schools throughout Elementary School, regarding both each school year and the increasing positive performance of schoolchildren due to course of the schooling process.Beside characterizing each group, it also shows its differences, which were already widely acknowledged and discussed in previous studies, as well as in national and international assessments (5,6,11) .
On the subject of the time spent to correctly decode the stimuli, the fact that the first year of public school often presents half of the number of children than private schools indicates that there is a significant deficit in the learning of grapheme-phoneme correspondences, which can result from different scenarios.It is paramount to emphasize that BP is a medium transparency language, with a greater transparency as to the grapheme for the phoneme relationship than the other way around (21) .In this perspective, some studies have shown that the delay in mastering the decoding skill in more transparent languages may be related to factors extrinsic to the child, such as the teaching methods, the family's schooling level and literacy practices, along with social vulnerability and/or deficits in emerging literacy skills (2,(11)(12)(13)(14)17) .
Therefore, it is important to consider the abovementioned variables in future studies so that the software is able to characterize and monitor the students' decoding, while simultaneously considering the individuals' socio-environmental aspects.This becomes crucial since a meta-analysis (24) aimed to verify the correlation between decoding performance at the beginning of literacy compared to at a later reading development showed evidence that decoding is a strong predictor of reading development throughout the academic life.
In other words, there is an undeniable demand for the early monitoring of children's decoding skill development through a fast, democratic instrument available in a software format for wide access, such as PRADE offers.This way, it will be possible to elaborate parent-teacher training programs and collective strategies aiming to, by analyzing the response curve to such activities, attenuate the influence of the extrinsic variables, allowing for an efficient identification of children with intrinsic difficulties and their subsequent referral for specific evaluation, as stated in the DSM-5-TR guidelines (25) .
With regard to the correct answer percentage and reading accuracy, the same phenomenon was observed, with a superior performance of private school children in both stimuli, for the different lengths and school years.It is worthy of mention, however, that such differences tend to display a decrease from the third school year onwards, as already pointed out in previous studies (10,17,24,26) , contradicting Brazil's new literacy policy, the National Commitment for Literate Children (27) , which recommends that Brazilian students should master basic decoding skills by the end of the second school year.Some authors argue that the greatest development in oral reading fluency measures has been observed between the first and third years of schooling, unlike what occurs between the third and fifth years, when there is a propensity for less variation in the assessment results (10,12,17,26) , data which was corroborated by the present study.
It should be emphasized that this development pattern finds theoretical foundation in the dual-route theory (development of automaticity of reading).Thus, as the individual learns to decode and master the written code, their performance on reading tests improves, that is, as their spelling lexicon increases, the scores related to the decoding speed tend to stabilize, with a propensity to reduce the differences among close schooling peers (1,10,13,17) .
The results found herein show similar development curves for all students, with improvement in all variables due to the school progression.This data confirms the cognitive science's hypothesis of reading that states the importance of learning and consolidating decoding in the first school years to reach reading fluency and comprehension, which is the final goal of this skill acquisition (5,6,13,14) .It is vital to stress, however, the need for further research concerning different types of schools in different regions of Brazil for the data to be generalized, which is the next step in PRADE's validation process.It is of fundamental importance to equally consider the performance of all students according to the stimulus length, regardless of the school type, since a strong effect of this variable was observed throughout all school years.These data are consistent with previous studies (11,26) that highlighted the influence of the stimulus length on the decoding performance of Brazilian schoolchildren, once again evidencing PRADE's effectiveness regarding the validity based on response processes.
A study postulated (28) that decoding assessment measures, when used as effective markers and predictors of future reading performance, are of great value to researchers, as well as to school staff, to professionals who assess reading development and its disorders, and also to public education agencies that monitor children's performance.The researchers carried out a meta-analysis in order to investigate whether the scores considered as a cutoff for possible reading development alterations were accurate enough to perform the proper measurement of this skill in children.The results of the analysis of journals, dissertations, theses and public documents indicated a great accuracy of different studies' cutoff scores for the identification of risk related to reading acquisition, reaffirming the decoding assessment as a valid tool for identifying potential delays in reading development (4)(5)(6)(10)(11)(12)(13) .
Finally, the literature specialized in the area along with the results found herein not only corroborate and support the data found by PRADE, but also indicates that there is a demand for instruments that enable a swift, accurate and democratic evaluation of the acquisition and development of decoding throughout the early school years, which is PRADE's main purpose.

CONCLUSION
The validity evidence stage based on the response processes of PRADE software is satisfactory in terms of characterizing different groups of the test's target population, adequately characterizing children from public and private schools according to their school level as well as in relation to the schooling effect in the process of the decoding acquisition for all students.Therefore, this instrument is assertive and ready for the next steps of validation.Software para avaliação e monitoramento do desenvolvimento da decodificação em escolares do Ensino Fundamental I: validade baseada nos processos de resposta

Figure 1 .
Figure 1.PRADE home screen displaying the welcome message and instructions

Figure 2 .
Figure 2. Example of how stimuli appear in PRADE

Table 1 .
Descriptive scores and comparative analysis of school types in relation to the correct word reading time according to word length, school and word types

Table 2 .
Descriptive scores and comparative analysis of school types in relation to the correct answer percentage according to word length, school year and word type

Table 3 .
Descriptive scores and comparative analysis of school types in relation to the correct answer percentage according to word length, school year and word type Caption: SD: Standard deviation; Min.: Minimum; Max: Maximum Máximoacordo com o ano escolar, extensão da palavra e tipo de estímulo, palavra/não-palavra.Os dados indicam maior número de crianças do primeiro ano de escola privada que conseguiram efetivamente decodificar palavras e não-palavras, diminuindo-se o número de participantes de escola pública em função do tamanho do estímulo.